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NASA’s Constellation Program employs a strategic analysis methodology in providing an 
integrated analysis capability of Lunar exploration scenarios and to support strategic 
decision-making regarding those scenarios. The strategic analysis methodology integrates 
the assessment of the major contributors to strategic objective satisfaction - performance, 
affordability, and risk - and captures the linkages and feedbacks between all three 
components. Strategic analysis supports strategic decision making by senior management 
through comparable analysis of alternative strategies, provision of a consistent set of high 
level value metrics, and the enabling of cost-benefit analysis. The tools developed to 
implement the strategic analysis methodology are not element design and sizing tools. 
Rather, these models evaluate strategic performance using predefined elements, imported 
into a library from expert-driven design/sizing tools or expert analysis. Specific components 
of the strategic analysis tool set include scenario definition, requirements generation, mission 
manifesting, scenario lifecycle costing, crew time analysis, objective satisfaction benefit, risk 
analysis, and probabilistic evaluation. Results from all components of strategic analysis are 
evaluated a set of pre-defined figures of merit (FOMs). These FOMs capture the high-level 
strategic characteristics of all scenarios and facilitate direct comparison of options. The 
strategic analysis methodology that is described in this paper has previously been applied to 
the Space Shuttle and International Space Station Programs and is now being used to 
support the development of the baseline Constellation Program lunar architecture. This 
paper will present an overview of the strategic analysis methodology and will present sample 
results from the application of the strategic analysis methodology to the Constellation 
Program lunar architecture. 


I. Introduction 

S trategic analysis serves to inform decision-makers of the benefit, viability, affordability, and robustness of 
system design options by providing integrated analysis of system performance over the hall system life cycle, 
from the cradle to beyond the grave. As NASA and partner agencies, both U.S. and international, move forward with 
the planning for human lunar return, strategic analysis will play a key role in helping to select architectures and 
scenarios that will result in productive, reliable, and affordable lunar exploration systems. A structured and robust 
approach to strategic analysis is required to ensure that these high-level planning and analysis activities take place in 
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a thorough and rigorous manner. In support of these goals, the Constellation Program Office’s (CxPO’s) Strategic 
Analysis Team (SAT) has developed a strategic analysis methodology and related tool set that allows for evaluation 
of exploration scenarios that integrates performance, affordability, and risk analysis. 

Over the past three years, NASA’s Constellation Program (CxP) senior management has applied this strategic 
analysis methodology and tool set to inform the design of lunar transportation and surface architectures, seeking 
refinement of the NASA Space Exploration Program through ongoing analysis to meet the needs, goals, and 
objectives defined by NASA’s Exploration Systems Mission Directorate, allowing for the achievement of the Vision 
for Space Exploration outlined by the President in 2004. The transportation and delivery of resources to support a 
permanent human presence will involve substantially more risk and cost than current/past Earth-orbiting, sustained 
systems, i.e. the International Space Station (ISS). The analysis methodology, therefore, merges deterministic and 
probabilistic approaches in an integrated manner to provide a broad trade space exploration capability and assess 
system robustness to unplanned, although not always unexpected, events. Cost-benefit analysis results of the 
expected, integrated system behavior are aggregated into high-level value metrics and figures of merit (FOMs) that 
serve as communication medium to inform decision-makers of the expected impacts and criticality of decisions. 

This paper will provide an overview of the strategic analysis methodology applied by CxP’s SAT. Individual 
discussions of the components of strategic analysis (performance, affordability, and risk) and an explanation of the 
macro- and micro-logistics concepts are provided. An example lunar exploration scenario that focuses on initial 
outpost build-up and continued human presence ties strategic analysis theories to results generation capabilities. The 
FOMs used to evaluate and compare lunar exploration scenarios are described, with that description including an 
application of those FOMs to the example lunar exploration scenario. Sensitivity analysis of the example scenario is 
also presented. The example scenario that is presented in this paper is notional and not representative of NASA’s 
official position on lunar exploration. 

II. Integrated Analysis Methodology Approach 

Strategic analysis focuses on integrating performance, affordability, and risk analysis and capturing the linkages 
and feedbacks between these three areas. A series of linked tools have been developed to support this integrated 
analysis. Specific tools include: a deterministic scenario definition tool, a scenario risk model, a probabilistic 
analysis tool, an affordability methodology, an evaluation of sustainability of various scenarios, and a micro- 
logistics model. Figure 1 displays the strategic analysis flow diagram, which represents the order of when the tools 
are utilized along with the data flow between the tools. An explanation of the concept of macro- and micro-logistics 
is presented along with a detailed discussion of each tool. 



Figure 1. Strategic Analysis Flow Diagram. 
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A. Macro- versus Micro-Logistics Concept 

Most exploration system analysis has focused on the delivery of elements and goods to a destination. This 
method, referred to within this paper as macro-logistics, is essential for understanding the high level behavior of 
human lunar exploration systems. It is particularly important because historically for ISS mission planning, one-half 
to two-thirds of overall transportation system delivery capacity is taken up by logistics and the carriers necessary to 
hold the logistics during the transfer to and storage at their final destination, leaving significantly reduced capacity 
for delivery of additional exploration hardware. Logistics consists of pressurized goods (crew consumables and 
spares & maintenance), unpressurized spares & maintenance, gases, and liquids that are required to keep the crew 
alive and elements functioning. The deterministic scenario definition tool and the probabilistic analysis tool both 
represent macro-logistics approaches. 

Focusing only on macro-logistics in this manner neglects a large portion of the constraints that will apply to a 
given exploration scenario. Micro-logistics refers to the local handling of goods at a location and must be studied in 
addition to macro-logistics to assess overall architecture performance 1 . Micro-logistics covers a broad range of 
topics, including storage requirements for goods, storage and movement of logistics carriers, the transfer of goods 
between carriers, the availability of goods for use by the crew, and the disposal of waste products. For example, as 
carriers are emptied over time during the course of an exploration scenario, the carriers can be emptied entirely and 
disposed of in some cases, but at other times the carriers are needed as containers for collecting and storing waste 
until they have been filled again. The micro-logistics tool is used to study these topics. 

The strategic analysis methodology integrates micro- and macro-level analysis of exploration scenarios, allowing 
for the evaluation of a number of issues that are not possible with analysis that is performed only at the macro-level 
alone. Specific issues can include, but are not limited to, storage time of goods on the surface versus the goods’ 
lifetimes, crew time requirements for transferring goods and carriers, availability of goods for normal operations or 
emergency situations, and determination of appropriate storage space for different items within a surface habitat. 

B. Deterministic Scenario Definition 

A candidate exploration scenario is initially defined using deterministic manifesting methodologies and tools, 
with that initial performance definition serving as the foundation for the more detailed probabilistic affordability, 
risk, performance, and value assessments. Development of the scenario definition involves an iterative process of 
mission definition and analysis, used to structure a complete scenario that nominally meets scenario requirements, 
with the basic phasing and structure of scenario definition being driven by scenario objectives and expectations from 
study leadership. The deterministic model relies on two basic types of input. Specific technical performance data for 
scenario elements and transportation options, developed by design teams and subject matter experts, are imported 
into the model to form the basis of a scenario. These elements and transportation options are fixed values and are not 
varied within a given scenario. Users then input specific options regarding how the scenario is structured to develop 
a specific scenario option. The parameters necessary to describe the set of missions that will constitute the scenario 
include: the number of crew delivered, the length of crewed surface duration, the delivery capacity of the 
transportation system, and the payloads (elements and logistics) delivered. Once the base scenario structure has been 
defined, the logistics necessary to sustain crew and element operations are calculated 2 and loaded into carriers for 
delivery prior to their intended date of use. If all requirements cannot be met, the structure of the scenario is 
modified and the calculation repeated. 

Scenario definition involves the evaluation of performance in a number of key areas. Specific areas of analysis 
evaluated in the scenario definition include: concept of operations; delivery requirements for crew, logistics, and 
utilization; mission mass and volume restrictions; eclipse considerations; power requirements and generation; and 
crew time availability. 

There are two types of volume restrictions - the first one is the volume restrictions of the elements and the 
second is the volume restrictions of the lunar lander. Certain elements such as the habitats and pressurized logistics 
modules can be used to deliver pressurized goods to the lunar surface. These elements can only deliver as many 
pressurized goods as can fit within a mass limit of the lunar lander and a volume limit of the element. The volume 
restrictions on the lunar lander are defined by the launch vehicle shroud and the allocated locations of where cargo 
can be placed on the lunar lander. Initially pictures and dimensions of the elements are used to make decisions on 
whether elements can be placed together on a lunar lander. For example, a habitat and a pressurized logistics module 
would not be loaded together on the lunar lander due to the length of the elements and the maximum dimensions of 
the cargo deck of the lunar lander. 

Power restrictions are driven by the number of power elements located on the lunar surface and the demand load 
of the elements located on the lunar surface at any given time. Each power system located at the Outpost can supply 
a fixed amount of power during the daylight and eclipse portions of the surface stay. As more elements are delivered 
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that require power, more power systems would also need to be delivered to account for the additional demands. 
Delivery of the power systems must be balanced with the delivery of the other elements and the overarching 
requirements for the lunar exploration scenario. 

The eclipse season must also be accounted for, not only factoring into the power needs, but the survival of the 
crew and other elements. Given an outpost located at the Lunar South Pole, approximately eight months out of the 
year are in constant daylight whereas the other four months of the year contain eclipses. A “blackout” period is 
defined as the period with an extensive number of back-to-back eclipses. When planning the scenario manifest, the 
avoidance of blackout periods might be desired if insufficient energy storage capability for extensive crewed 
operations is delivered. The missions would need to be organized to make good use of the daylight periods. The 
missions should be phased to have at least one crew mission for every daylight period, with the crewed mission 
being launched near the beginning of the daylight period to maximize the schedule margin within that daylight 
period. Cargo missions may be landed during the blackout period if there is sufficient power on the lunar lander to 
support the cargo until the cargo is removed from the lunar lander. In addition, any element at the Outpost would 
have to be designed to, at a minimum, survive the eclipse season even if the Outpost is not inhabited during these 
times. 

A concept of operations must also be considered. For example, if cargo landers are launched during the eclipse 
season and their cargo cannot be removed until the crew is present on the lunar surface, provisions must be included 
to ensure the cargo survives until removed. This would include power, thermal, and logistics considerations. 
Another example is the removal of the habitats from the lunar lander. If the assumption is that the habitats would be 
connected on the lunar surface, then an element would be required to remove the habitats from the lunar lander and 
transport to the final destination. A large payload transport would need to be delivered to the Outpost prior to the 
connecting of the habitats. These are examples of concept of operations that need to be considered when 
determining the delivery order of the elements to the Outpost. 

The non-calculation intensive nature of deterministic scenario definition makes the application of the approach 
ideal for broad trade space exploration, with the exploration often limited not by approach complexity or time. 
Satisfaction of requirements through an iterative analysis cycle can be quickly completed, allowing a large number 
of candidate scenarios to be rapidly developed and evaluated. Sensitivity analysis of certain variables coupled with 
particular trade studies can also be easily conducted to characterize the nominal scenario behavior, particularly when 
focused on key system parameters, such as different physical characteristics of the elements, their associated 
logistics, required crew consumables, and the logistics packaging methodology. 

C. Risk Analysis 

The strategic analysis methodology and tool set includes an estimation of scenario risk, including both safety and 
uncertainty. On a mission-by-mission basis, the risk evaluation tools have the capability to estimate the probability 
of different possible mission end states. Based on vehicle reliability, the tool uses an event-tree analysis to simulate 
each stage of a mission and then uses the probability of failure at each stage to determine the overall likelihood of 
failure. The model aggregates the results of the event-tree to determine the likelihood that the mission will result in 
various different possible modes of failure, including loss of crew and loss of cargo. 

The vehicle reliability model that is integrated in the risk model includes an analysis of vehicle maturity. Based 
on the mission definition and a specified test program, this component of the tool estimates the reliability of each 
transportation element for each specific flight, taking into account the experience and reliability growth that has 
accumulated prior to that mission. 

A separate component of the risk model has the capability to evaluate launch probabilities in multi-launch 
scenarios. Because many exploration scenarios include missions that launch multiple vehicles that rendezvous in 
low Earth orbit (LEO), there is a risk to the mission that all vehicles may not be able to launch in a timely manner. 
Based on vehicle orbital lifetimes and historical launch probability data, the multi-launch analysis tool estimates the 
probability that all vehicles will launch in time. 

Certain results from the risk model are reported out as independent FOMs. These parameters include the 
probability of loss of crew and the probability of loss of key surface architecture elements. In addition, the risk 
results are also used to drive the probabilistic analysis. 

D. Probabilistic Analysis 

History has shown that complex, integrated space systems rarely operate according to a pre-defmed plan. The 
stochastic nature of such systems results in unplanned, although not unexpected, events occurring that result in 
modifications of the planned concept of operations. To assess a given lunar scenario’s robustness and to develop 
operational policies for contingency situations, the uncertainty and risk factors that contribute to the system 
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stochasticity need to be assessed in an integrated manner. Probabilistic scenario analysis is used to simulate a large 
number of “possible” scenario outcomes, based upon the likelihood of occurrence of certain events and a set of pre- 
determined contingency rules. The results of this analysis are distributions for expected scenario results. These 
distributions can then be compared to the nominal value to evaluate the robustness of the scenario to adverse events 
and to test and optimize contingency planning. 

Within each simulated scenario run, the probabilistic scenario analysis tool performs a mission-by-mission 
temporal simulation. At each mission step, the tool uses the deterministic scenario definition tools to calculate a 
planned manifest for all remaining missions, including requirements, capacities, and loadings. The outcome of the 
current mission is then simulated based on probability distributions for all possible non-nominal events and mission 
event trees. Once the outcome of the mission has been determined, if the mission is successful, the tool tracks the 
additional material that is delivered to a site on the lunar surface and the amount of material that is consumed. In this 
manner a running inventory of surface deliverables is maintained. The consumption of material on the lunar surface 
can also be driven by probabilistic data. Failures of equipment, use of logistics, and crew activity rates can be 
represented stochastically. If the current mission experiences a failure, then the consequences and resultant delays to 
the remaining scenario missions are determined, based upon specified contingency operational policy. The 
remaining flights are reset based upon these consequences. 

The tool then moves on to the next flight in the scenario and repeats the simulation. This flight, and all the flights 
that follow, are therefore influenced by the events that have occurred cumulatively on all previous flights. After all 
the flights in a scenario have been simulated, the overall scenario performance for that case is evaluated. The 
amount of potential science conducted, the extensibility objectives that are met, additional costs that are incurred, 
and the risk to the crew are determined. 

The probabilistic scenario tool repeats this process many times, simulating a large number of possible scenario 
outcomes and collecting performance data for each. The performance data is then integrated into probabilistic 
distributions for expected scenario results. These distributions show the likelihood of achieving different levels of 
scenario performance based on the current reliability, control policies, and uncertainties within the system. The 
probability distributions can be compared to the nominal scenario performance, as predicted in the scenario 
definition tools, to evaluate the robustness of the given scenario. 

Scenarios that provide a high level of expected performance across the range of possible probabilistic outcomes 
are identified as being more robust. That is, they are relatively insensitive to the real-world events that disrupt 
planned behavior. Scenarios that exhibit a sharp drop-off in expected performance are less robust. 

Based on the results of the probabilistic analysis, revised scenario definitions may be developed to provide 
additional robustness against adverse events and to optimize contingency planning to better ensure a high level of 
expected scenario performance. Typically, however, in order to improve the expected performance under 
probabilistic conditions, it is necessary to sacrifice some level of nominal performance. Nominal performance is 
typically traded for increased robustness through increased redundancy, contingency deliveries, schedule margin, or 
other mitigation techniques. 

Probabilistic analysis tools allow mitigation techniques to be optimized and can demonstrate the ultimate values 
of these measures to decision-makers, who otherwise will tend to focus on nominal performance measures. This 
additional insight into mitigation of critical failures and the implications for the planned scenario and its associated 
logistics support necessitate the inclusion of probabilistic analysis when defining a scenario 3 . 

E. Affordability 

Affordability analysis incorporates both the integration and manipulation of various cost estimates into a holistic 
purview along with comparison to forecasted available budgets. The intent is to understand from a strategic 
standpoint what impact various design decisions will have on expected costs while acknowledging the inherent 
uncertainty associated with cost estimates and future budgetary considerations. To that end, affordability analysis is 
handled from a conceptual level systems engineering perspective inclusive of trade comparisons, what-if scenarios, 
and other methods of understanding various sensitivities and options. Affordability analysis is also graphical in 
nature in that results are provided via traditional time-phased area charts (i.e., “sandcharts”), cumulative distribution 
function (c.d.f.) curves, as well as cumulative plots that are incorporated into FOM analyses (Section IV). 

7. Life Cycle Analysis 

In order for analysis to include all germane affordability considerations, the entire life cycle cost (LCC) of 
various lunar exploration elements is required (with the sole exception of disposal costs since end of scenarios can 
be open-ended). In similar nature to the scenario definition tool, affordability analysis does not include actual 
element cost estimating and analysis. Such work typically takes place within separate element design teams. 
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Although actual cost integration work takes place in parallel with affordability analysis, discussion here is limited to 
the actual affordability assessment portion and not integration activities such as cost estimate assumption 
consistency checks, master WBS development and coordination, and element-level estimate sanity checks. Since 
element-level cost estimates often do not include full life cycle costs, particularly those costs that are more manifest- 
driven in nature, affordability analysis also pulls from other supporting analysis capabilities such as the inclusion of 
sustaining engineering and spares costs from a modeling capability at the Jet Propulsion Laboratory (JPL) titled the 
“Exploration Architecture Operations Cost Model” (ExAOCM) 4 . A top-level summary of costs that are typically 
included is demonstrated along with the corresponding phase definition 5 in Table 1. 


Table 1. Life Cycle Cost Considerations. 


Life Cycle Consideration 

Type 

Program Phase 5 

Conceptual Design 

Element 

Pre-Phase A and Phase A Costs 

Design, Development, Test, and Evaluation 

Element 

Phase B/C Costs 

Production (for single unit) 

Element/Manifest 

Phase D Costs 

Sustaining Engineering 

Manifest 

Phase E Costs 

Operations Support 

Manifest 

Phase E Costs 

Spares & Logistics 

Manifest 

Phase E Costs 

Government Oversight 

All 

All 


2. Modeling Overview 

Affordability analysis modeling is handled both deterministically and probabilistically in order to enable rapid 
trade space analyses and uncertainty quantification for certain cases. 

Deterministic analysis, performed using the Integrated Affordability Analysis Tool (IAAT), focuses on 
integrating the data together to enable the addition/removal of lunar exploration elements, easy modification of 
element manifest dates, and manipulation of other driving assumptions to allow full trade space exploration. The 
deterministic results (i.e., point estimates) provide tremendous insight into the behavior of candidate lunar 
exploration scenarios and their contributing costs. 

Probabilistic analysis leverages the expertise within Johnson Space Center’s (JSC) Program Planning and 
Control (PP&C) Office, who developed a model based on IAAT using Tecolote Research Inc.’s Automated Cost 
Estimating Integrated Tools (ACEIT) environment 6 . Select cases identified via deterministic trade space exploration 
are analyzed using ACEIT to allow the generation of c.d.f. curves representing total cost uncertainty associated with 
a scenario trade option through a point in time (typically through fiscal year 2020). 

3. Driving Affordability Parameters 

The use of affordability analysis within the context of strategic analysis seeks to provide understanding into the 
advantageous trade options, the viable strategic decisions, and affordability against given forecasted budgets. 

Scenario Definition Manipulation - Typically the largest impact on lunar exploration scenario affordability 
pertains to the scheduling of development activities of various scenario elements. Multi-element scenarios by nature 
require several development paths which have to be planned within finite budgetary constraints. If additional monies 
are not expected during a constrained time then schedule assumptions have to be manipulated. The modeling 
framework enables such manipulation by allowing the element delivery manifests to be easily modified. All life 
cycle considerations, including development, are directly tied to manifest dates, so this type of trade, performed in 
conjunction with the deterministic scenario definition tool, allows optimization of spending profiles relative to 
budgets. 

Element Manipulation - In addition to exploring various manifest schedule scenarios for a given set of scenario 
elements, actual addition and removal of elements is often warranted. Master Work Breakdown Structures (WBSs) 
are used, but the modeling capability provides easy manipulation of which elements are included within the analysis. 
A master element list exists that contains cost characteristics of potential elements, so the incorporation of any given 
deterministic scenario definition file automatically identifies which elements are included, preventing onerous model 
“set-up” time. 

Budget Scenarios - Since actual Agency budgets do not stretch beyond a near-term horizon, assumptions have to 
be made as to the expected behavior of out-year available dollars. These assumptions themselves become scenario- 
driven since they are often based on other programs and projects. For instance, the planned retirement of Shuttle in 
FY10 and the subsequent retirement of the International Space Station both have direct budgetary impact on 
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available dollars for CxP. The affordability analysis modeling capability handles this interaction by allowing for 
“budget” adjustments that modify the baseline budget per varying scenarios. 

Percent Contribution - The final major parameter relates to potential collaboration between NASA and other 
entities; either commercial or international. Historical examples of collaboration have demonstrated significant cost 
savings when an outside entity develops an element for inclusion, with NASA still paying for incorporation and 
operation. Since the actual cost to NASA can vary significantly depending on the circumstances, such cost is treated 
parametrically usually as a percentage of original estimated cost if NASA had developed the element itself. To that 
end, all major cost roll-ups provide factors that can easily be manipulated to appropriately model collaboration 
scenarios. 

F. Sustainability 

Because lunar exploration scenarios are likely to occur over an extended period of time, public and political 
sustainability are key areas of performance that must be evaluated as part of the strategic analysis. Perceived value 
for the NASA Space Exploration Program can be equated to level of interest (LOI) to the public, businesses, and 
politicians. The evaluation of programmatic sustainability involves three interactions (see Figure 2): 

1 . The LOI that a program produces over time and a comparison of that level to an identified acceptable limit 

2. How any deficits in LOI will translate into reductions in funding (or possible changes in funding due to 
other factors) 

3. The ability of the program to continue to produce value when subjected to potential changes in funding 

Analysis of an exploration scenario 

sustainability should primarily involve 
evaluation of interaction #1, which involves 
analyzing the production of perceived value 
over time and comparing that value to an 
established minimum. Interaction #2 should be 
relatively consistent between scenarios. 

Interaction #3 involves uncertainty related to 
external changes to the budget. 

The methodology and tool set developed 
by the SAT have the capability to evaluate the 
sustainability of an exploration scenario. The 
approach used measures the perceived output 
of a scenario and compares that to some 
minimal acceptable limit. First, a ‘benchmark 
event’ is established that defines LOI required to sustain budget (e.g. Spirit & Opportunity Landing). A nominal LOI 
weight for that event is defined (Spirit and Opportunity are assigned a value of five). An LOI weight is then assigned 
to each potential scenario event based on the relative LOI that it will generate (e.g. first human landing is assigned 
an LOI of ten, third human landing is assigned an LOI of one). A reasonable ‘decay rate’ is set, where decay rate is 
defined as the rate that the interest dissipates. A histogram of the LOI over the scenario is used to visualize the 
calculated interest. The histogram is compared to the benchmark minimal limit, with the area under the limit 
representing the threat to sustainability. 

G. Micro-Logistics 

The strategic analysis methodology and tool set includes a set of models designed to evaluate the micro-logistics 
of an exploration scenario 7 . As described above, micro-logistics analysis concerns the handling, storage, usage, and 
disposal of goods at a destination. 

In order to properly evaluate micro-logistics constraints and to determine to what degree those constraints might 
limit ultimate productivity at a human lunar outpost, the SAT developed a micro-logistics model that has the 
capability to track the delivery, positioning, usage, and disposal of all goods at the Outpost site. 

The micro-logistics model is comprised of two separate analysis modules and a visualization tool. The two 
analysis modules simulate micro-logistics at a human lunar outpost over time using a system dynamics 
methodology. One analysis module tracks the delivery, storage, and conversion of all liquids and gases at the 
Outpost site. The second module tracks the delivery, storage, usage, and disposal of all solid goods at the Outpost. 
The visualization tool provides for interpretation and analysis of scenario results in an interactive manner. 

The micro-logistics model is run against a standardized scenario description, which is produced by the lunar 
exploration analysis tools employed by the SAT. The scenario description specifies the mission schedule, crew 


External Changes 
to Budget 
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Figure 2. Interactions of Programmatic Sustainability. 
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durations, and element delivery for a specific scenario to be evaluated. In addition, the scenario description captures 
the manifesting plan for each mission, as developed by the scenario definition tools for that scenario. This 
manifesting plan specifies the carriers loaded on each flight and the specific types and amount of cargo in each 
carrier. The micro-logistics model imports the scenario description, and, based on input parameters, produces a time- 
based simulation of all micro-logistics activities at the Outpost. 

III. Example Scenario 

Strategic analysis is used to analyze lunar exploration scenario options and to determine the benefits and 
consequences of those options. An example scenario is presented to demonstrate the application of the strategic 
analysis methodology and to show example results. The FOMs in Section IV and the sensitivity analysis in Section 
V are based on this example scenario. The example lunar exploration scenario presented in this section is notional 
and is not representative of NASA’s official position on lunar exploration. The results of the example scenario are 
focused on initial outpost buildup and achieving continued human presence. The primary assumptions established 
for the example scenario include: 

2019 start date, maximum of 4 missions per year 
Two crewed missions per year, starting in 2021 
Outpost location at Lunar South Pole 
Emphasis on early outpost buildup 
Maximum crewed duration of 1 80 days 
500 kg of science delivered for each mission 

Before the example scenario is discussed, the transportation and lunar surface systems elements used in this example 
are defined. 

A. Transportation and Lunar Surface System Element Definitions 

The transportation system 8 elements used in this example consist of two launch vehicles, Ares I and Ares V, an 
element that carries the crew, Orion, and a lunar lander, Altair. The lunar surface system elements in this example 
consist of elements that provide habitation, power, mobility, communication, carriers, and In-situ Resource 
Utilization (ISRU). 

The Ares I launch vehicle consists of a 5-segment reusable Solid Rocket Booster (SRB) and a J-2X upper stage. 
The Ares I delivers the crewed Orion vehicle to low Earth orbit (LEO). The Ares V launch vehicle consists of two 
5.5-segment reusable SRBs, a core stage with RS-68 engines, and an Earth Departure Stage (EDS). The Ares V 
delivers the Altair lunar lander to LEO. Once in LEO, the Altair and partially used EDS rendezvous and dock with 
the Orion. Then, the EDS is utilized to perform the Trans-lunar injection burn. The Altair descent stage performs the 
Low Lunar Orbit (LLO) injection burn, inserting the Orion and Altair into LLO. The crew transfers from Orion to 
Altair and descends to the lunar surface using the Altair while the Orion remains uncrewed in LLO. At the end of 
the surface stay, the Altair ascent stage transports the crew to LLO to rendezvous with Orion. Orion then performs 
the maneuvers to return the crew to Earth. The Altair assumed performance to Lunar South Pole is 14.6 t payload in 
a cargo mode, 0.5 t payload in a crewed sortie mode, and 1.0 t payload in a crewed outpost mode (increase primarily 
from removal of the airlock). The crewed sortie mode supports a crew of four for up to seven days on the lunar 
surface and contains an airlock for crew ingress/egress. The crewed outpost mode supports a crew of four for the 
duration of transport between LLO and the lunar surface and back to LLO. While on the lunar surface, residual 
propellant in the lunar lander descent stages can be scavenged to generate 400 kg of water per lunar lander. 

The lunar surface system elements primarily support longer duration crew presence and extend mobility ranges. 
The habitation cluster (Hab-1, Hab-2, and Hab-3) provides the main living space for the crew while on the lunar 
surface. Hab-2 and Hab-3 also partially serve as pressurized logistics modules to transport pressurized goods to 
support the crew during the early missions. The pressurized logistics modules (PLM) are sized to maximize 
commonality with the habitation structure. The power systems (PS) consist of solar arrays and fuel cells to support 
crewed surface stays through the eclipse seasons. Power systems are sent with habitats and PLMs until sufficient 
power is available on the lunar surface to support the elements needs. A mobile power system (MPS) is also 
included to support long range traverse from the Outpost to locations of interest. The MPS would be used in 
conjunction with one or more Pressurized Rovers (PR). A pressurized rover consists of a pressurized crew cabin on 
top of a mobility chassis (MC). A pressurized rover can transport a crew of two within 100 km of the Outpost. The 
MC allows for transportation of smaller elements or crew. The MC can be used with the Crew Driving Kit (CDK) to 


8 http://www.nasa.gov/mission_pages/constellation/main/index.html 
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form an unpressurized rover, which can transport crew up to 10 km from the Outpost. Mobility Chassis Tools 
(MCT) are delivered to the Outpost and can be attached to the MC. These include a winch, tool interface, and 
manipulator work package. 

The Large Payload Transport (LPT) can remove large elements, such as habitats and PLMs, from the lunar 
lander and place them into their predetermined locations in the Outpost layout. The Offloading and Transportation 
Support Equipment (OTSE) aids in removing elements and logistics from the lunar lander. An ISRU plant and tools 
are available to produce oxygen from the lunar regolith. The Communications Terminal (CT) allows for short-range 
and medium-range communications between the Outpost, elements, direct-to-Earth, and communications satellites. 
All these elements are included in the example scenario and support humans living on the Moon. 

B. Outpost-centric Scenario 

Figure 3 shows the deterministic scenario definition for the example scenario, with Human Lunar Return (HLR) 
in fiscal year (FY) 2019. Only the surface system elements and pressurized logistics modules are shown in the 
figure. The unpressurized goods, gases, and liquid and their associated carriers and tanks are not shown for clarity. 
As seen in the figure, the delivery of habitation in 2020 allows for successive crews to stay longer on the lunar 
surface than a standard sortie mission of seven days length. The elements are also placed on specific missions to 
obtain a balance between the capabilities the elements provide and the logistics required to support the crew for a 
given number of days. The figure only shows the missions through 2026 since the latter flights would be repeated at 
a rate of three missions per year for three years followed by four missions per year for one year to sustain continued 
human presence. 
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Figure 3. Lunar Exploration Mission Description for Example Scenario. 

IV. Strategic Level Figures of Merit 

The strategic analysis methodology and tool set is designed to rapidly produce a standardized set of Figures of 
Merit (FOMs) for any given scenario that is evaluated. FOMs are used to evaluate and compare the relative merits of 
differing Lunar Exploration scenarios, approaches, and executions. The FOMs should be discrete enough to compare 
relative value expected to be achieved by closely related exploration systems (i.e. capable of evaluating differences 
in delivered mass, crew time, etc.). The set of high-level FOMs should provide a comprehensive view of the merits 
of the scenario across all relevant areas of merit. These FOMs should be as concise as possible to facilitate 
understanding by decision-makers and direct comparison of scenarios. The FOMs should be as mutually 
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independent as possible. In addition, the FOMs should be as non-redundant as possible to ensure that each metric is 
measuring a unique feature of the system. For the Lunar Exploration analyses, the comprehensive set of high-level 
FOMs that is used covers five major areas: Affordability, Benefit, Safety & Mission Assurance, Programmatic Risk, 
and Sustainability. Each of these is discussed in the following sections. 

A. Affordability 

Affordability as a strategic figure of merit provides decision makers an overall indication as to how projected 
cost estimates compare to projected available budget dollars. In essence there are two constraints that have to be met 
for a given scenario option; affordability as a total cumulative cost and affordability associated with cost phasing. 
Not only do there have to be enough monies to cover the full life cycle cost through a certain point in time, but 
within any given year during that time the dollars available are required to cover the content amount estimated. The 
Affordability FOM shown in Figure 4 provides information on both types of constraint. NASA may not be 
constrained by these constraints because of international or commercial involvement. 

The tabulated data within the Affordability FOM provides three types of metrics. The “Total Budget Delta to 
2020, RY$B” provides the total overage/shortage of an architecture option’s estimated life cycle cost relative to an 
available budget. This total is a comparison of the total value and does not take into account the time-phased nature 
of the costs and budget. To account for such, the “Max Annual Deficit, RY$B” metric demonstrates the maximum 
shortage for any given year. The third metric, “Max Cumulative Difference, RY$B” combines both the life cycle 
roll-up of the first metric with the annual shortage consideration of the second metric since it captures the annual 
shortages cumulatively while also allowing overage years to mitigate the shortages. Instead of just knowing what the 
maximum annual shortage is, the maximum cumulative difference indicates the magnitude of debt expected due to 
multiple years of shortages, thus providing insight into the appropriateness of the proposed phasing. The third metric 
also makes a subtle assumption that budget dollars left over in years where an overage exists can be “piggy-banked” 
for later use. 

The graphical portion of the Affordability FOM within Figure 4 captures the annual differences via time-phased 
stacked columns. Red columns represent years where a shortage exists (also indicated by the negative value) while 
black columns represent years where there is adequate budget for projected costs. Taken together with the three 
metrics captured within the table, a full representation of a candidate scenario can be portrayed. Due to the sensitive 
nature of cost data, the table has been filled in with “TBD” (to be determined) and the graph is notional. 
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Figure 4. Affordability Figure of Merit for Example Scenario through First Ten Years of Exploration. 


B. Benefit FOMs 

Benefit FOMs measure the total worth or value produced by Lunar Exploration scenarios across all themes of 
interest. These themes are based on the Global Exploration Strategy six exploration themes and objectives 9 10 : 

1. Exploration Preparation: Reduce the risks and increase the productivity of future missions in our solar 
system by testing technologies, systems, and operations in an off-Earth planetary environment. 

2. Scientific Knowledge: Engage in scientific investigations of the Moon, on the Moon, and from the Moon. 

3. Human Civilization: Develop the knowledge, capabilities, and infrastructure required to live and work on 
the Moon, with a focus on continually increasing the number of individuals that can be supported on the 
Moon, the duration of time that individuals can remain on the Moon, and the level of self-sufficiency of 
lunar operations. 

4. Economic Expansion: Create new markets, based on lunar and cis-lunar activity that will return economic, 
technological, and quality-of-life benefits to all humankind. 


9 http://www.nasa.gov/mission_pages/exploration/mmb/why_moon.html 
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5. Global Partnership: Enhance global security by providing a challenging, shared, and peaceful global vision 
that unites nations in collaborative pursuit of common objectives. 

6. Public Engagement: Use a vibrant exploration program to excite the public about space, encourage students 
to pursue careers in high technology fields, and ensure that individuals enter the workforce with the 
scientific and technical knowledge necessary to sustain exploration. 

These themes were developed by NASA to answer the question of why the global space community should return to 
the Moon. For the purposes of the Lunar Exploration analyses, in order to better match resource utilization and 
avoid overlaps in these areas, these themes have been re-categorized into the following areas: Extensibility & 
Experience, Science & Lunar Survey, Economic, Global Partnership, and Public Engagement. Extensibility & 
Experience includes certain objectives from Exploration Preparation and Human Civilization. Science & Lunar 
Survey includes certain objectives from Exploration Preparation, Scientific Knowledge, and Economic Expansion. 
Economic includes the remaining Economic Expansion objectives. Global Partnership and Public Engagement are 
direct translations that include all objectives from those areas. Extensibility & Experience and Science & Lunar 
Survey are discussed in the following sections. Objectives in the areas of Economic and Public Engagement require 
minimal resources and it was assumed that these objectives would be accomplished for all scenarios. Since the 
objectives are not discriminators between scenarios, FOMs were not explicitly stated for these areas. 

1. Extensibility and Experience 

Experience is a measurement of time and activities on the surface, which include surface days, first human 
landing, and capability for continuous human presence. Extensibility is a measure of lunar exploration objectives 
that are met. Together, the Extensibility & Experience FOMs measure accomplishment in Exploration Preparation 
and Human Civilization, which include three objective areas: 1) development, testing, and demonstration of relevant 
technologies, processes, and components for extensibility to future exploration; 2) accumulated experience in living 
off the Earth, maintaining equipment, and performing useful exploration; and 3) accumulated experience in living on 
the Moon. 

The Extensibility & Experience FOMs are shown in Figure 5. Key milestone dates for the scenario are 
highlighted with the first crewed mission date and the first 180-day surface stay day. Both planned dates and 
“expected” dates are shown, where the expected date incorporates the probabilistic analysis results. The example 
scenario has an expected delay of four months for the first crewed mission and approximately one-half year for the 
1 80-day capability. Lunar experience is represented by the number of crewed missions and the days on surface. Of 
the 21 missions planned, eighteen are expected to be successfully completed by 2029. In addition, only 70% of the 
number of days on surface is expected to be achieved by 2029. The percentage of extensibility objectives met is also 
displayed. Most of the extensibility objectives are met deterministically and probabilistically. The graph displays the 
cumulative histograms of the planned and expected days on surface, along with highlighting the planned key 
milestone dates and the planned crewed mission and location. For this example scenario, all the missions landed at 
the Outpost location, whereas other scenarios might explore alternate locations with sortie or super sortie missions. 



Figure 5. Extensibility & Experience Figures of Merit for Example Scenario through First Ten Years of 
Exploration. 


As seen in Figure 5, the percentage of extensibility objectives met does not change between the planned and 
expected values. This is due to the ability to satisfy the majority of extensibility objectives with the successful 
completion of the first several 180-day stay periods, which in of themselves are highly likely to occur whether 
measured deterministically or probabilistically. Figure 6 details the completion of extensibility objectives for the 
example scenario, including a full listing of all the objectives and an indication as to what degree each objective is 
satisfied as the scenario progresses. 
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Figure 6. Extensibility Objective Satisfaction for Example Scenario. 
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2. Science and Lunar Survey 

Science & Lunar Survey FOMs measure accomplishment in Exploration Preparation, Scientific Knowledge, 
Human Civilization and Economic Expansion. These themes contribute to Science & Lunar Survey in four objective 
areas: 1) conduct of fundamental science; 2) science conducted to support future exploration; 3) science/survey 
conducted to support future lunar exploration; and 4) science/survey conducted to determine opportunities for 
commercial endeavors. Capability for Science & Lunar Survey is measured through the potential number of the sites 
visited and the resources available at those sites. These resources include mass for science delivered to the surface, 
crew time available for scientific activities, Intra-vehicular Activity (IV A), and Extra-vehicular Activity (EVA), and 
mobility capability at each site. The Science & Lunar Survey FOMs are shown in Figure 7. The planned science 
mass to the surface is typically set as a ground rule for analysis on a per mission basis. The science, IVA, and EVA 
time is determined from the total amount of productive crew IVA and EVA time on the surface minus time required 
to complete other basic activities, including training, logistics, planning, public engagement, and maintenance time. 

The expected values for these two quantities take into account the probability analysis results. The number of 
sites visited and the mobility radius at each site is also called out. For outpost-centric scenarios (as with the example 
scenario), the number of sites is typically one. When sorties are included in the scenario, the number of sites 
increases by the number of different sortie locations explored. The mobility radius is determined by the type of 
elements that are delivered for a scenario. For example, if no mobility is available for use the assumption is 0.5 km, 
and for two unpressurized rovers the mobility radius is 25 km, whereas for two pressurized rovers the mobility 
radius increases to 100 km. The example scenario includes delivery of two pressurized rovers; therefore, the 
mobility radius is 100 km. The graph displays cumulative histograms of planned science mass delivered, expected 
science mass delivered, planned science & IV A/EVA hours, and expected science & IV A/EVA hours. 
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Figure 7. Science & Lunar Survey Figures of Merit for Example Scenario through First Ten Years of 
Exploration. 


C. Safety and Mission Success 

Safety & Mission Success FOMs measure expected losses of the system. Safety FOMs capture the expected 
losses that are due to uncertainty or reliability. These include the expected loss of life and expected loss of missions. 
The primary Safety FOM measures total expected human loss. Mission Success FOMs capture expected losses to 
mission critical elements such as habitation and power elements. The current risk model utilized is based only on 
transportation system risk drivers and therefore, the expected losses to mission critical elements are only due to the 
loss of transportation elements, not a failure of the actual element. A model to calculate the probability of failure for 
the surface elements is under development. Figure 8 presents the probabilities of losing one or more crew and one or 
more key elements. The habitation and power systems are displayed since these elements are required to support 
crew survival on the lunar surface. The graph shows the mission by mission probability of loss of crew (PLOC). The 
missions that have no PLOC reported are cargo only missions. The PLOC decreases over the first few missions due 
to the engine maturation. 
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Figure 8. Safety & Mission Success Figures of Merit for Example Scenario through First Ten Years of 
Exploration. 


D. Programmatic Risk 

Analysis of programmatic risk involves the evaluation of the likelihood and consequence of changes in the 
performance of the scenario due to multiple types of programmatic uncertainty such as vehicle performance, 
technology development, schedule, budget, and mission reliability. Presently, only mission reliability is being 
evaluated. A probabilistic engine is utilized to develop probability distributions for achieved values. A Monte Carlo 
analysis is conducted based on input distributions for key uncertainties. The scenario of interest is adjusted based on 
realized events. The results of the Monte Carlo are integrated to calculate expected values. The metrics that feed into 
the probabilistic engine are probability of loss of crew, probability of loss of mission, probability of an anomaly 
occurring, probability of missed rendezvous, expected delay periods, and contingency plans. Programmatic Risk 
FOMs measure expected values for other FOMs and are not presented independently but are shown in comparison to 
the planned value for each FOM. Robust scenarios are those that will produce high relative value across a wide 
range of probabilistic conditions. 

E. Sustainability 

The Sustainability FOM measures the ability of a program to produce a level of value over time (or perceived 
value) that justifies continued investment in the program. Perceived value for the Lunar Exploration Program can be 
equated to level of interest to the public, businesses, and politicians. 

The approach used to develop the Sustainability FOM is to measure the perceived output of a scenario and 
compare that to some minimal acceptable limit. A histogram of the LOI over the scenario is calculated based on the 
events that occur in that scenario and the expected public perception of those events. The histogram is compared to a 
benchmark minimal limit and the area under the limit represents a threat to sustainability. As seen in Figure 9, the 
example scenario LOI falls below the minimum threshold 78% of the scenario duration. However, for this example 
scenario, there was no intent to optimize the latter missions in order to improve the Sustainability FOM. 



Figure 9. Sustainability Figures of Merit for Example Scenario through First Ten Years of Exploration. 


V. Sensitivity Analysis 

Sensitivity analysis allows for the exploration of trade space variables to understand what variables affect the 
system and how significant of an impact the variable has on a scenario. Single- variable and multi-variable sensitivity 
analyses should be performed to understand the key drivers in a system. Variables include phasing of element 
delivery, level of technology performance, ISRU production capacity, etc. The examples in the following section are 
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single variable sensitivities focusing on the lunar lander payload capacity and the delivery of science. Primary 
method to achieve closure in the sensitivity analyses is to increase or decrease crew surface days. Based on study 
guidelines, the specific elements for all the sensitivity analysis were not resized to optimize the scenario. In general, 
the delivery order of the elements was maintained. However, in some cases, elements had to be reordered to allow 
for sufficient logistics to be delivered to support the crew. To fully take advantage of (or mitigate) a change in 
performance, the scenario must be fully restructured, with the elements being resized to optimize the overall 
performance of the scenario. 

The primary metric used to compare sensitivity cases is planned cumulative surface stay days. This metric allows 
for rapid analysis turnaround (deterministic only) while serving as a first order measure for comparison of scenarios. 
The secondary metric is unallocated lunar lander capacity. Often, Lunar Exploration scenarios are distinctly 
different, but have similar planned cumulative surface stay days. This secondary metric is used to differentiate 
between those scenarios. This metric is the total unallocated delivery mass over the entire scenario, which is 
available to bring additional hardware. This mass cannot be used to increase cumulative surface stay days because of 
imposed constraints on the scenario. Potential options for utilizing the unallocated delivery mass include delivery of 
additional science payloads, mobility, advanced power systems, etc; and not to use the available mass to add 
robustness for unforeseen mass growth. 

A. Lander Payload Capacity 

The baseline assumption for lunar lander payload capacity is 14.6 t in a cargo mode, 0.5 t in a crewed sortie 
mode, and 1.0 t in a crewed outpost mode. This sensitivity analysis explores the deterministic results of the example 
scenario when varying the payload capacity of the lunar lander, which directly translates to variations in 
transportation system performance to a lunar polar outpost. These sensitivities of potential lunar lander capacity 
focus on the capability of the element to impact the scenario but do not reflect actual changes to the element design. 
The crewed sortie mode payload of 0.5 t was not varied. The crewed outpost mode and cargo mode were varied in 
0.5 t increments such that the crewed outpost mode payload ranged from 0 t to 2 t. The corresponding cargo lunar 
lander mode payload is shown in Figure 10 along with the results of the analysis. 
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Figure 10. Lander Payload Capacity Sensitivity Results. 

The planned cumulative surface stay days are shown as two groups - days through 2023 and days after 2023. 
Recall in Figure 3, the first 180-day stay was in 2024 followed by crewed and cargo missions to support continued 
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human presence at the Outpost. Therefore, to show the primary effects of increasing and decreasing lunar lander 
payload capacity, the data was compiled to look at effects during outpost buildup and during continued human 
presence. As seen in Figure 10, the increase in lunar lander capacity can increase the number of surface days through 
2023 by approximately 30% (85-95 days). However, as the lunar lander capacity is decreased, the number of surface 
days through 2023 decreases by approximately 65% (204 days). In addition, many elements had to be shifted in the 
deterministic scenario definition in order to achieve a minimum of 14 days of stay duration for each crewed mission. 
This shifting of elements reduces some of the capabilities at the Outpost and some of the initial redundancy in the 
system. The crew days after 2023 were only impacted by the payload capacities when the payload capacity was 
decreased to 0.0 t for crewed outpost mode (13.6 t cargo mode). Most of this impact was driven by the inability to 
carry elements or logistics on a crewed outpost mode lander, whereas all the other cases allowed for this possibility 
even if not fully utilized. In addition, the unallocated lander capacity allows for logistics to be shifted between the 
landers to better balance the missions. 


B. Science Delivered 

Sensitivity analysis can also be used to explore sensitivity in policy. The baseline assumption for the science 
delivered for the example scenario is that 500 kg of science is delivered for each mission (crew or cargo), meaning 
that the science is available for that mission date; however, the science can be delivered to the lunar surface on or 
before (pre-positioned) that mission date. The sensitivity analysis explored delivery of science with three other 
assumptions - on each crew and cargo mission (no pre-positioning), for each crewed mission (pre-positioning okay), 
and on each crewed mission (no pre-positioning). Figure 1 1 displays the results of the science delivered sensitivity. 
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Figure 11. Science Delivered Sensitivity Results. 


As seen in Figure 11, requiring science to be delivered on every mission (whether it be crew and cargo or crew 
only) reduces the number of resulting surface days due to less efficiency in packaging of the science carriers and the 
lunar landers. Pre-positioning of science also allows for the delivered science package mass to be greater than the 
500 kg per mission allocation for the science packages. Requiring 500 kg to be delivered on every mission 
potentially places an artificial constraint on the scientific community whereas pre-positioning could allow larger 
science packages to be emplaced on the lunar surface. The crew days after 2023 did not change because additional 
pressurized goods would need to be delivered to support an increase in surface days. Each cargo lunar lander is 
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already delivering a pressurized logistics module and there were no additional cargo lunar landers added to the 
scenario definition to enable delivery of additional pressurized goods. 


VI. Conclusions 

As NASA evaluates options for returning to the Moon, a methodology must be employed that allows for a 
structured and robust analysis and comparison of high-level figures of merit for Lunar Exploration scenarios. The 
described strategic analysis methodology allows for this type of comparative evaluation to occur through an 
integrated analysis of scenario performance, affordability, and risk. 

Strategic analysis provides decision-makers with a high-level tool to evaluate a broad range of design options 
and provides an environment in which strategic decision-making can occur in a structured, consistent, and robust 
environment. The strategic analysis methodology and tool set that are being employed to evaluate lunar exploration 
scenarios help propagate detailed technical performance, cost, and reliability data for various design options up to a 
higher level, allowing for the strategic impact of these options on high-level NASA objectives to be evaluated. This 
allows decision-makers to base design choices on a broad set of figures of merit, measured against strategic 
objectives, rather than focusing solely on the performance of individual elements. 

In addition, strategic analysis allows for decision-making to be expanded beyond just selection of elements and 
specification of a nominal scenario to the development of operational policy and contingency planning. A broad 
analysis of the operational aspects of scenario options will allow for the development of a lunar exploration scenario 
that not only provides high value across a broad range of strategic objectives but also provides for high-levels of 
robustness and sustainability. 
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